Eyetracker Mounts for Use with Handheld Devices

ABSTRACT

A device for performing eyetracking on a handheld device includes and eyetracking camera and an eyetracking camera boom mount. The eyetracking camera boom mount physically and electrically connects a handheld device and the eyetracking camera. The eyetracking camera boom mount includes an extension boom that positions the eyetracking camera behind the user&#39;s hands. The extension boom provides the eyetracking camera with a view of the user&#39;s eyes that is unobstructed by the user&#39;s hands. The device can further include an operating scene camera for monitoring a person&#39;s hand operations on the handheld device. The operating scene camera can be mounted on the same extension boom as the eyetracking camera or on a separate extension boom.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.14/726,327, filed May 29, 2015, which claims the benefit of U.S.Provisional Patent Application No. 62/005,859, filed May 30, 2014, thecontent of all of the above is incorporated by reference herein in theirentireties.

INTRODUCTION

When a person is operating a handheld device, such as a smart phone ortablet computer, it is sometimes desired to track the person's gazepointon and/or around the device. A key design problem in this gaze trackingapplication is the positioning of the eyetracking camera. In particular,it is important to allow the user full, normal operation of his hands,fingers and thumbs, while not obstructing the eyetracking instrument'sview of the eyes, or obstructing the user's view of the device and itsdisplay. As a result, mounting methods for attaching eyetrackinginstruments to handheld devices are needed to overcome these conflictsbetween continuous eyetracking and full, natural hand motions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a typical hand held computer device with a display screen,a user's eye looking at the display screen and a camera mounted abovethe display screen that may be configured to observe the user's eye.

FIG. 2 shows a hand held computer device where the eyetracking camera ismounted below the display screen and mounted on an extension boom thatpositions the camera behind the user's hands and provides the camerawith an unobstructed view of the user's eye or eyes, in accordance withvarious embodiments.

FIG. 3 illustrates a handheld computer device where the eyetrackingcamera is mounted on a flexible extension boom, allowing the orientationof the eyetracking camera to be adjusted to point at the user's eye, inaccordance with various embodiments.

FIG. 4 illustrates a handheld computer device where an additionaloperating-scene camera, positioned adjacent to the eyetracking camerabehind the use's hands, monitors both the device screen area and theuser's fingers and hands, providing feedback about the user hand/fingeroperations as the user operates the handheld device, in accordance withvarious embodiments.

FIG. 5 is a block diagram that illustrates a computer system, inaccordance with various embodiments.

FIG. 6 is a schematic diagram showing an eyetracker, in accordance withvarious embodiments.

FIG. 7 is a schematic diagram showing an eyetracker that includes aneyefollower, in accordance with various embodiments.

Before one or more embodiments of the invention are described in detail,one skilled in the art will appreciate that the invention is not limitedin its application to the details of construction, the arrangements ofcomponents, and the arrangement of steps set forth in the followingdetailed description or illustrations. The invention is capable of otherembodiments and of being practiced or being carried out in various ways.Also, it is to be understood that the phraseology and terminology usedherein is for the purpose of description and should not be regarded aslimiting.

DETAILED DESCRIPTION Computer-Implemented System

FIG. 5 is a block diagram that illustrates a computer system 500, uponwhich embodiments of the present teachings may be implemented. Computersystem 500 includes a bus 502 or other communication mechanism forcommunicating information, and a processor 504 coupled with bus 502 forprocessing information. Computer system 500 also includes a memory 506,which can be a random access memory (RAM) or other dynamic storagedevice, coupled to bus 502 for storing instructions to be executed byprocessor 504. Memory 506 also may be used for storing temporaryvariables or other intermediate information during execution ofinstructions to be executed by processor 504. Computer system 500further includes a read only memory (ROM) 508 or other static storagedevice coupled to bus 502 for storing static information andinstructions for processor 504. A storage device 510, such as a magneticdisk or optical disk, is provided and coupled to bus 502 for storinginformation and instructions.

Computer system 500 may be coupled via bus 502 to a display 512, such asa cathode ray tube (CRT) or liquid crystal display (LCD), for displayinginformation to a computer user. An input device 514, includingalphanumeric and other keys, is coupled to bus 502 for communicatinginformation and command selections to processor 504. Another type ofuser input device is cursor control 516, such as a mouse, a trackball orcursor direction keys for communicating direction information andcommand selections to processor 504 and for controlling cursor movementon display 512. This input device typically has two degrees of freedomin two axes, a first axis (i.e., x) and a second axis (i.e., y), thatallows the device to specify positions in a plane. Yet another type ofuser input device is a touch screen, allowing the user to touch thescreen to select or activate items or functions displayed on the screen.An additional type of input device may be an eyetracker, including aneyetracking camera that looks at the user's eye(s) and means forcalculating where on the screen the user is looking, allowing the userto select or activate items or functions displayed on the screen bylooking at them.

A computer system 500 can perform the present teachings. Consistent withcertain implementations of the present teachings, results are providedby computer system 500 in response to processor 504 executing one ormore sequences of one or more instructions contained in memory 506. Suchinstructions may be read into memory 506 from another computer-readablemedium, such as storage device 510. Execution of the sequences ofinstructions contained in memory 506 causes processor 504 to perform theprocess described herein. Alternatively hard-wired circuitry may be usedin place of or in combination with software instructions to implementthe present teachings. Thus implementations of the present teachings arenot limited to any specific combination of hardware circuitry andsoftware.

In various embodiments, computer system 500 can be connected to one ormore other computer systems, like computer system 500, across a networkto form a networked system. The network can include a private network ora public network such as the Internet. In the networked system, one ormore computer systems can store and serve the data to other computersystems. The one or more computer systems that store and serve the datacan be referred to as servers or the cloud, in a cloud computingscenario. The other computer systems that send and receive data to andfrom the servers or the cloud can be referred to as client or clouddevices, for example.

The term “computer-readable medium” as used herein refers to any mediathat participates in providing instructions to processor 504 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, volatile media, and transmission media.Non-volatile media includes, for example, optical or magnetic disks,such as storage device 510. Volatile media includes dynamic memory, suchas memory 506. Transmission media includes coaxial cables, copper wire,and fiber optics, including the wires that comprise bus 502.

Common forms of computer-readable media or computer program productsinclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, digital videodisc (DVD), a Blu-ray Disc, any other optical medium, a thumb drive, amemory card, a RAM, PROM, and EPROM, a FLASH-EPROM, any other memorychip or cartridge, or any other tangible medium from which a computercan read.

Various forms of computer readable media may be involved in carrying oneor more sequences of one or more instructions to processor 104 forexecution. For example, the instructions may initially be carried on themagnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 500 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detectorcoupled to bus 502 can receive the data carried in the infra-red signaland place the data on bus 502. Bus 502 carries the data to memory 506,from which processor 504 retrieves and executes the instructions. Theinstructions received by memory 506 may optionally be stored on storagedevice 510 either before or after execution by processor 504.

In accordance with various embodiments, instructions configured to beexecuted by a processor to perform a method are stored on acomputer-readable medium. The computer-readable medium can be a devicethat stores digital information. For example, a computer-readable mediumincludes a compact disc read-only memory (CD-ROM) as is known in the artfor storing software. The computer-readable medium is accessed by aprocessor suitable for executing instructions configured to be executed.

The following descriptions of various implementations of the presentteachings have been presented for purposes of illustration anddescription. It is not exhaustive and does not limit the presentteachings to the precise form disclosed. Modifications and variationsare possible in light of the above teachings or may be acquired frompracticing of the present teachings. Additionally, the describedimplementation includes software but the present teachings may beimplemented as a combination of hardware and software or in hardwarealone. The present teachings may be implemented with bothobject-oriented and non-object-oriented programming systems.

Eyetracker and Eyefollower

In general, an eyetracker or an eyegaze system is a device or instrumentthat is used to determine where an eye is looking. Modern eyetrackers,sometimes referred to as video eyetrackers, are camera-based devicesthat observe a person's eyes and predict the point in space where theperson is looking. This point in space is referred to as the gazepoint,for example. The line connecting the fovea of the eye, the center of theeye pupil, and the gazepoint is referred to as the gaze line, forexample.

FIG. 6 is a schematic diagram showing an eyetracker 600, in accordancewith various embodiments. Eyetracker 600 includes camera 610,illumination source 620, and processor 630. Illumination source 620illuminates eye 640, and camera 610 images eye 640. Processor 630receives the image from camera 610 and determines the position and focuscondition of eye 640 within the camera image. Based on the position andfocus condition of the eye within the camera image, the processor isable to compute the 3D location and orientation of actual eye 640 withinthe camera body frame of reference.

Eyetracker 600 can include additional elements. For example, eyetracker600 can include one or more additional cameras (not shown) or one ormore additional optical devices (not shown) to determine the range fromcamera 610 to eye 640. Eyetracker 600 can also include a display (notshown) to determine the gazepoint in an image displayed by processor 630on the display.

An important objective of many eyetrackers is to allow the user to movehis head freely while the eyetracker continues to track the user's gazewith high accuracy. Typical head motions involve moving (translating)the head from side to side, up and down, and back and forth; and involverotating the head forward and back (pitching or nodding), rotating theface from left to right (yawing or shaking), and rotating the headtoward one shoulder or the other (rolling). One method for minimizinghead motion with respect to an eyetracking device is to place theeyetracker device on the user's head, attached to a pair of glasses, forexample. In many applications, however, it is desired to position theeye eyetracking device at a remote, off-head location. Accommodatinghead motion with respect to the eyetracker platform is particularlyrelevant to the objective of capturing high quality, high resolution eyeimages in remote eyetrackers.

To accommodate variable positions and orientations of the head withrespect to the eyetracker platform, eyetrackers may include mechanizeddevices to keep the eyetracking camera(s) physically pointed at and/orfocused on the user's eye(s). A motorized pan-tilt device, such as agimbal, may be used, for example, to keep the camera's view directionpointed at the eye or eyes; and a focus motor may be used to keep theeye(s) in focus. Eyetrackers that utilize such mechanical means toautomatically point or focus their eyetracking cameras on the eyes aresometimes referred to as eyefollowers.

FIG. 7 is a schematic diagram showing an eyetracker 700 that includes aneyefollower, in accordance with various embodiments. Eyetracker 700includes camera 710, camera-pointing gimbal 715, and processor 720. Theeyefollower portion of eyetracker 700 includes, for example, a motorizedmechanical configuration comprising gimbal 715 to keep the camerapointed at the user's eyes and a motorized lens (not shown) for camera710 that implements a variable-focus range. The eyefollower portion ofeyetracker 700 may also include a motorized zoom capability (not shown)for camera 710 that implements a variable zoom. Gimbal 715 controls theyaw (pan) and pitch (tilt) of camera 710, which, in turn, are used tofollow a user's eye 740 as the user moves his head and/or eye 740 fromside to side and up and down, respectively. The motorized lens forcamera 710 is used to follow the user's eye 740 as the user moves hishead forward or backward, and it may also be used in the computation ofthe range from the camera to the eye based on a measurement of the lenslength required to put the eye in focus. The motorized zoom for camera710 may be used to control the desired size, i.e., pixel resolution, ofthe eye image within the camera image as the user moves his head forwardand backward. The camera-pointing gimbal 715 can be optionally securedto display 730, mechanically linking the coordinate system of camera 710to display 730, for example. Note that camera 710 can also include anillumination source (not shown).

Processor 720, for example, executes control loop algorithms required tokeep the camera pointed at, focused on, and zoomed on eye 740 as theuser moves his head. Processor 720 also executes, for example,algorithms used to solve rigorous trigonometric gazepoint trackingequations, referred to as “explicit raytrace algorithms.” These explicitraytrace algorithms are used to accurately predict the user's gazepoint750 on display 730, fully accommodating the variable camera geometry andthe moving head and/or eye 740.

Eyetracker Camera Mounts for Use with Handheld Devices

As noted above, when a person is operating a handheld device it issometimes desired to track the person's gazepoint on and/or around thedevice. A key design problem in this gaze tracking application is thepositioning of the eyetracking camera. In particular, it is important toallow the user full, normal operation of his hand(s), finger(s) andthumb(s), while not obstructing the eyetracking instrument's view of theeyes, or obstructing the user's view of the device.

In various embodiments, methods for attaching eyetracker instruments,such as an eyetracking camera and/or an operating-scene camera, tohandheld devices are provided to overcome these conflicts betweencontinuous eyetracking and full, natural hand motions.

FIG. 1 is a schematic diagram 100 of a handheld device that includeseyetracking, in accordance with various embodiments. As illustrated inFIG. 1, the eyetracking camera 120 could be mounted at the top of thehandheld device 101, above the device's display screen 110. Thisconfiguration would allow the eyetracker camera 120 what might seem tobe an unobstructed view 130 of the user's eye(s) 150 while allowing theuser free motion of his hands to operate the device naturally—withoutthe hand(s) obstructing the eyetracker camera's view of the user's eyes.A significant advantage of the eyetracker camera 120 being located abovethe display screen 110 as shown in the FIG. 1 is that the camera 120 maybe embedded within the main body of the handheld device 101, minimizingthe overall size and dimensions of the handheld device 101. (In FIG. 1,the eye's gaze line is indicated by dashed line 160, and the user'sgazepoint on the display is indicated by the dot 170.)

A key shortcoming of mounting the eyetracking camera 120 above thedisplay 110, however, is that when a person is looking below aneyetracking camera, his upper eyelids can often block a sufficientlycomplete view of the eyeball required for the eyetracking camera toaccurately measure the eye orientation. Thus, for reliable eyetrackeroperation, it is not sufficient to simply provide the camera with a goodview of the eye region of the user's face; it is also necessary toprovide a sufficient view of the eyeball itself, unobstructed by theeyelids. (In pupil-center/corneal reflection (PCCR) method eyetrackers,for example, it is necessary for the eyetracker to see both the pupiland the corneal reflection in order to compute the eye's angularorientation, and thus to compute the eye's gazepoint.) To allow a morecomplete view of the eyeball, with less obstruction from the eyelids, itis common, as illustrated in FIG. 7, to place the eyetracking camerabelow the screen the user is viewing.

When the screen is a small handheld device, however, rather than alarger computer screen, embedding the eyetracker camera both a) withinthe main body of the device and b) below the display area, results in asignificant operational problem: the user's hands, in the normaloperation of the handheld device, can easily obstruct the camera's viewof the eyes.

FIG. 2 is a schematic diagram 200 showing an eyetracker extension boommount 240 for use with a handheld device 201, in accordance with variousembodiments. Eyetracking camera 220 is located below the screen ofhandheld device 201. Eyetracker extension boom mount 240 is attached atone end to the bottom end of handheld device 201, below display 210, andattaches at the other end to eyetracking camera 220. Eyetracker boommount 240 positions eyetracking camera 220 behind the user's hands,providing the eyetracker camera with a view of the user's eyes that isunobstructed by the user's hand(s). Eyetracking camera 220 pointstowards the user's eyes 250. In addition to keeping the user's handsfrom obscuring the eyetracking camera's view 230 of the user's eyes 250,this embodiment minimizes the camera's physical interference of theuser's hand operations and allows the user an unobstructed view of thedevice 201 and its display 210. In FIG. 2, the eye's gaze line isindicated by dashed line 260, and the user's gazepoint on the display210 is indicated by the dot 270. In addition to mechanically (orphysically) connecting the eyetracking camera 220 to the handheld device201, extension boom mount 240 may provide electrical and electronicconnections between eyetracking camera 220 and handheld device 201. Insome cases, information transferred between eyetracking camera 220 andhandheld device 201 may be transmitted wirelessly.

In various embodiments, an eyetracker boom mount for use with a handhelddevice 1) provides the eyetracking camera an unobstructed view of theuser's eyes, allowing continuous eyetracking, 2) allows the user to movehis handheld device around freely, 3) allows the user to physicallyaccess the handheld device freely with either or both hands, 4) providesminimum interference of the user's natural hand operations of thehandheld device, and 5) maintains the user's full vision of the handhelddevice and its displays.

Flexible Booms

FIG. 3 is a side view 300 of an eyetracker boom mount, in accordancewith various embodiments. As illustrated in FIG. 3, extension boom mount340 can be made of any material that is flexible, allowing the pointingorientation or observed view 330 of the eyetracking camera 320 to beadjusted to point at the user's eye 350. Alternatively, the extensionboom mount 340 can be a semi-rigid arm made of, for example, semi-rigidmaterials. One skilled in the art will appreciate that other types ofmaterials can be used for the extension boom connecting the eyetrackingcamera/operating-scene camera 320 and the handheld device 310.

In various embodiments, a flexible extension boom may contain sensors orinstrumentation to measure the position and/or orientation of theeyetracking camera with respect to the handheld device and/or itsdisplay. Thus, for example, as a flexible boom is moved and the positionand/or orientation of the eyetracking camera moves with respect to thehandheld device and/or its display, the user's gazepoint on the handhelddevice and/or display may be calculated accurately from the eyetracker'simage of the user's eye(s) based on the known position and orientationof the eyetracking camera(s), despite the movement of the eyetrackingcamera(s) with respect to the handheld device. For example, coordinatetransformation methods, well known in the field of eyetracking opticalray tracing, may be used to accommodate changes in eyetracking cameraposition and orientation with respect to the handheld device. (“ExplicitRaytracing for Gimbal-Based Gazepoint Trackers,” U.S. Pat. No.7,686,451.)

Operating-Scene Camera Mounts for Use with Handheld Devices

When a person is operating a handheld device, it is sometimes desired tomonitor his hand activity with respect to the display and/or touchscreen. Observing hand activity may be useful, for example, indetermining the usability of the device itself, evaluating the usabilityof an application running on the device, or monitoring the user responseto situations presented by an application program being run on thehandheld device. In this specification, a camera that monitors a) theuser's hand operations (including hand, finger and thumb movements), b)the device display, and/or c) the area around the device's display, istermed an “operating-scene camera,” or simply a “scene camera.” Invarious embodiments, operating-scene cameras can be implemented onhandheld devices either independently of or in conjunction witheyetracking cameras.

FIG. 4 is a side view 400 of a handheld device 401 equipped with both anoperating-scene camera 480 and an eyetracker camera 420. Boom mounts 440and 460 position both eyetracking camera 420 and operating-scene camera480 behind the user's hand(s) (not shown). In contrast to theeyetracking camera 480 facing in the direction of the user's eye(s) 450,the operating-scene camera faces toward the handheld device and towardthe user's hand(s). The field of view of operating-scene camera 480 isindicated by the dashed lines 490. Similar to eyetracker boom mount 440in FIG. 4, operating-scene camera boom 460 is attached to the bottom ofhandheld device 401 below the display screen 410.

When it is desired to include an operating-scene camera for use with ahandheld device, operating-scene camera 480 may be attached to handhelddevice 401 in several configurations. In one embodiment (not shown),scene camera 480 may be mechanically attached directly to eyetrackingcamera 420, with the electrical connections to/from handheld device 401travelling through eyetracking camera 420 and eyetracker-camera boom440. This configuration has the constraint, however, that the relativeviewing directions of the scene and eyetracking cameras cannot beadjusted with respect to one another. In various embodiments, scenecamera 480 may be mounted on its own extension boom 460, providingseparate direction adjustments for the scene and eyetracking cameras.Extension boom 460 may originate directly (not shown) from handhelddevice 401 or emerge from eyetracker-camera boom mount 440, as shown inFIG. 4. Thus, in various embodiments, a single extension boom for botheyetracking camera 420 and operating-scene camera 480 is attached to thehandheld device 401, the boom then splitting, or dividing, into twobooms, or branches, one attached to eyetracking camera 420 and the otherattached to operating scene camera 480.

In various embodiments, operating-scene camera extension boom 460 may beflexible.

In various embodiments, operating -scene camera extension boom 460 maybe detachable from handheld device 401.

In various embodiments, operating -scene camera 480 may be connected tooperating-scene camera mount 460 through a motorized camera-pointinggimbal that allows operating-scene camera 480 to follow the movement ofthe user's hands.

Eyetracker Interfaces to Handheld Devices

In various embodiments, the handheld device 401 is a smartphone ortablet computer, for example. Many handheld devices, such as smartphones and tablet computers, contain computer technology that caninterface directly with and support eyetracking and eyefollowingapplication software. Recording and analysis of the user's eye and handactivity, for example, may be performed within the handheld deviceitself. Additionally, the eyetracking image processing software and/orthe eyefollower control software (that controls an eyefollower's gimbaland lens motors) can be executed on the handheld device processor,eliminating the need for duplicating that processing capability in theeyetracker instrument itself.

In various embodiments it may also be advantageous to utilize a cameraor cameras already built into the handheld device to support theeyetracking and face detection functions of the eyetracker. In someeyefollower configurations, for example, a face detection function,operating with a relatively wide field of view, is used to detect thelocation of the face and/or eyes within a larger volume of free headspace, and this detected face-location information is used to direct theeyetracking camera(s), which may have a significantly smaller field ofview, to point at and focus on the user's eyes, thus allowing theeyetracking camera(s) to commence their eyetracking functions. Theeyetracker's use of the handheld device's existing camera(s) caneliminate the need for including face-detection camera(s) in theeyetracker/eyefollower equipment itself.

In various embodiments, information about eyetracking andhand-operations may be transmitted wirelessly between the handhelddevice and remote data collection and analysis terminals.

Detachable Cameras

With many handheld devices, it may be desirable to use the device eitherwith or without an eyetracking camera, or with or without anoperating-scene/hand monitoring camera. For these cases, it is desirableto construct the cameras and their mounts such that they can be easilydetached from the handheld device, i.e. easily removed from andreattached to the handheld device. The attachment mechanisms must, ofcourse, accommodate both a) electronic connections to supportinformation communication between the device and the camera or camerasthe and b) mechanical connections to physically support the camera orcameras with respect to the handheld device.

Eyetrackers with Asymmetric Apertures

Some advanced eyetracking cameras employ asymmetric apertures thatenable eye image processing algorithms to precisely measure the range ofthe user's eye(s) from the eyetracking camera (U.S. Ser. No. 14/634,410,Improved Asymmetric Aperture for Eyetracking). In various embodiments ofeyetrackers mounted on handheld devices, the eyetracking camera maycontain an asymmetric aperture.

Eyefollowers on Handheld Devices

Though present day eyefollowers with motorized camera-pointing gimbalsand motorized lenses are practically too large to be attached tohandheld devices today, the use of micro-electro-mechanical systems(MEMS) in Eyefollowers may make such implementations far more feasiblein the future. In various embodiments of this invention, eyetrackerswith eyefollower capabilities, including a motorized camera-pointinggimbal and/or a motorized camera lens, may be attached to the handhelddevice booms, allowing the eyetracking camera to follow the user'seye(s) as the relative position of the user's head moves with respect tothe handheld device.

Closing Comments

Embodiments of systems and methods for implementing eyetracker mountsfor use with handheld devices are described in this detailed descriptionof the invention. In this detailed description, for purposes ofexplanation, numerous specific details are set forth to provide athorough understanding of embodiments of the present invention. Oneskilled in the art will appreciate, however, that embodiments of thepresent invention may be practiced without these specific details. Inother instances, structures and devices are shown in block diagram form.Furthermore, one skilled in the art can readily appreciate that thespecific sequences in which methods are presented and performed areillustrative and it is contemplated that the sequences can be varied andstill remain within the spirit and scope of embodiments of the presentinvention.

The foregoing disclosure of the preferred embodiments of the presentinvention has been presented for purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Many variations andmodifications of the embodiments described herein will be apparent toone of ordinary skill in the art in light of the above disclosure. Thescope of the invention is to be defined only by the claims appendedhereto, and by their equivalents.

Further, in describing representative embodiments of the presentinvention, the specification may have presented the method and/orprocess of the present invention as a particular sequence of steps.However, to the extent that the method or process does not rely on theparticular order of steps set forth herein, the method or process shouldnot be limited to the particular sequence of steps described. As one ofordinary skill in the art would appreciate, other sequences of steps maybe possible. Therefore, the particular order of the steps set forth inthe specification should not be construed as limitations on the claims.In addition, the claims directed to the method and/or process of thepresent invention should not be limited to the performance of theirsteps in the order written, and one skilled in the art can readilyappreciate that the sequences may be varied and still remain within thespirit and scope of the present invention.

What is claimed is:
 1. A device for performing eyetracking on a handhelddevice, comprising: an eyetracking camera; and an eyetracking cameraboom mount that physically and electrically connects a handheld deviceand the eyetracking camera, wherein the eyetracking camera boom mountincludes an extension boom that positions the eyetracking camera behindthe user's hands, providing the eyetracking camera with a view of theuser's eyes that is unobstructed by the users hand(s), wherein theextension boom provides the position and orientation of the eyetrackingcamera with respect to the handheld device and its display.
 2. Thedevice of claim 1, wherein the extension boom is attached to the bottomof the handheld device.
 3. The device of claim 1, wherein the extensionboom is flexible.
 4. The device of claim 3, wherein the flexibleextension boom is instrumented with sensors to the measure the positionand/or orientation of the eyetracking camera with respect to thehandheld device and/or its display.
 5. The device of claim 1, whereinthe eyetracking camera and the eyetracking camera mount are detachablefrom the handheld device.
 6. The device of claim 1, wherein theeyetracking camera contains an asymmetric aperture.
 7. The device ofclaim 1, wherein the eyetracking camera is attached to the eyetrackingcamera mount through a motorized camera-pointing gimbal and/or amotorized camera lens, allowing the eyetracking camera to follow theuser's eye as the relative position of user's head moves with respect tothe handheld device.
 8. A device for monitoring a person's handoperations on a handheld device, comprising: an operating-scene camera;and an operating-scene camera boom mount that physically andelectrically connects a handheld device and the operating-scene camera,wherein the operating-scene camera boom mount includes an extension boomthat positions the operating-scene camera behind the user's hands andprovides the operating-scene camera with a view of the handheld deviceand the user's hand(s) as he operates the handheld device, wherein theextension boom provides the position and orientation of the eyetrackingcamera with respect to the handheld device and its display.
 9. Thedevice of claim 8 wherein the operating-scene camera's boom mount isattached to the bottom of handheld device.
 10. The device of claim 8wherein the operating-scene camera's boom mount is flexible.
 11. Thedevice of claim 8 wherein the operating-scene camera is detachable fromthe handheld device.
 12. The device of claim 8 wherein theoperating-scene camera is connected to the operating-scene camera mountthrough a motorized camera pointing gimbal that allows theoperating-scene camera to follow the movement of the user's hands.